Note on stochastic control problems related with general fully 
coupled forward-backward stochastic differential equations 



Juan Li 

School of Mathematics and Statistics, Shandong University at Weihai, Weihai 264209, P. R. China. 

E-mail: juanli@sdu.edu.cn 

June 21, 2012 



Abstract. In this paper we study stochastic optimal control problems of general fully coupled forward- 
backward stochastic differential equations (FBSDEs). In Li and Wei [8] the authors studied two cases of 
diffusion coefficients a of FSDEs, in one case when a depends on the control and does not depend on the 
second component of the solution (Y, Z) of the BSDE, and in the other case a depends on Z and doesn't 
depend on the control. Here we study the general case when a depends on both Z and the control at the 
same time. The recursive cost functionals are defined by controlled general fully coupled FBSDEs, then 
the value functions are given by taking the essential supremum of the cost functionals over all admissible 
controls. We give the formulation of related generalized Hamilton- Jacobi-Bellman (HJB) equations, and 
prove the value function is its viscosity solution. 

Keywords. Fully coupled FBSDEs; value functions; stochastic backward semigroup; dynamic programming prin- 
ciple; algebraic equation; viscosity solution. 
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1 Introduction 



Pardoux and Peng [12] first introduced nonlinear backward stochastic differential equations (BSDEs) 
driven by a Brownian motion. Since then the theory of BSDEs develops very quickly, see El Karoui, 
Peng and Quenez [6], Peng [14], [15], [16], etc. Associated with the BSDE theory, the theory of fully 
coupled forward-backward stochastic differential equations (FBSDEs) develops also very quickly, refer to, 
Antonelli p], Cvitanic and Ma g], Delarue [5], Hu and Peng [7], Ma, Protter, and Yong [9], Ma, Wu, Zhang, 
and Zhang [10], Ma and Yong [II], Pardoux and Tang [33], Peng and Wu [H], Wu pi], Yong [22], [23], and 
Zhang [24], etc. For more details on fully coupled FBSDEs, the reader is referred to the book of Ma and 
Yong IT]; also refer to Li and Wei [8] and the references therein. 

Pardoux and Tang [T3] studied fully coupled FBSDEs (but without controls), and gave an existence 
result for viscosity solutions of related quasi-linear parabolic PDEs, when the diffusion coefficient a of the 
forward equation does not depend on the second component of the solution (Y, Z) of the BSDE. Wu and 
Yu [20], [21] studied the case when the diffusion coefficient a of the forward equation depends on Z, but, 
the stochastic systems without controls. In Li and Wei [8 they studied the optimal control problems of fully 
coupled FBSDEs. They studied two cases of diffusion coefficients a of FSDEs, that is, in one case a depends 
on the control and does not depend on and in the other case a depends on Z and doesn't depend on the 
control. They use a new method to prove that the value functions are deterministic, satisfied the dynamic 
programming principle (DPP), and were viscosity solutions to the related generalized Hamilton- Jacobi- 
Bellman (HJB) equations. The associated generalized HJB equations are related with algebraic equations 
when a depends on Z and doesn't depend on the control. They generalized Peng's BSDE method, and in 
particular, the notion of stochastic backward semigroup in [17] . When a depends on Z makes the stochastic 
control much more complicate and the related HJB equation is combined with an algebraic equation, which 
was inspired by Wu, Yu [20]. However, they use the continuation method combined with the fixed point 
theorem to prove very clearly that the algebraic equation has a unique solution, and, moreover, also give 
the representation for this solution. On the other hand, they also prove some new basic estimates for fully 
coupled FBSDEs under the monotonic assumptions. In particular, they prove under the Lipschitz and linear 
growth conditions that fully coupled FBSDEs have a unique solution on the small time interval, if the 
Lipschitz constant of a with respect to z is sufficiently small. They also establish a generalized comparison 
theorem for such fully coupled FBSDEs. Here we want to study the general case, that is, when a depends 
on Z and the control at the same time, for this general case what the associated HJB equations will be. 

Let us be more precise. We study a stochastic control problem related with fully coupled FBSDE. 
The cost functional is introduced by the following fully coupled FBSDE: 

' ,/.V:- : *' = 6(s,Y s t ^ ;M ,Y s t ^ ; ",Z^ ;M ,u s )ds + cr(s,Y s t ^ ;, \Y s t ' :l;;u ,Z^ ;u ,M s ) ( i J B s , 

,1)! ' " = -f{s 1 X t s ^ u 1 Y^ x ' u 1 Z t s ' x ' u ,u s )ds + Zl^ u dB s , se[t,T], 
' X t,x;u = x, ' 
k Y% x < u = $(Y^ ;M ), 

where T > is an arbitrarily fixed finite time horizon, B = (-B s )sg[o,t] is a d-dimensional standard Brownian 
motion, and u = (u s ) se [ t T] is an admissible control. Precise assumptions on the coefficients 6, a, /, $ are 
given in the next section. Under our assumptions, (|1.1|) has a unique solution (Xl ,X]U , Y^- X]U , Z t s > x ' u ) se \ t x\ 
and the cost functional is defined by 

J(t,x;u) =Yf< x < u . (1.2) 
We define the value function of our stochastic control problems as follows: 

W(t,x) := ess sup ueUt T J (t,x;u). (1.3) 

The objective of our paper is to investigate this value function. The main results of the paper state that W 
is deterministic (Proposition 2.1), continuous viscosity solution of the associated HJB equations (Theorem 
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3.1). The associated HJB equation is combined with an algebraic equation as follows: 
&W(t, x) + H v (t, x, W(t, x)) = 0, 

V(t, x, u) = DW(t, x).a(t, x, W(t, x), V(t, x, u),u), (t, x) G [0, T) x R™, u G U, (1.4) 
W(T,x) = <f>(x), iel". 

In this case 

H v (t, x, W{t, x)) = sup{DW.b(t, x, W(t, x), V(t, x, u),u) + \tr{acr T {t, x, W(t, x),V(t, x, u),u)D 2 W(t, x)) 

u£U 

+f(t,x,W(t,x),V(t,x,u),u)}, 

where t G [0,T],x G R". 

Our paper is organized as follows: Section 2 introduces the framework of the stochastic control prob- 
lems. In Section 3, we prove that W is a viscosity solution of the associated HJB equation described above. 



2 Framework 

Let (f2, T, P) be the classical Wiener space, where fi is the set of continuous functions from [0, T] to 
R d starting from (O = Co([0, T]; R d )), J- is the completed Borel a-algebra over fi, and P is the Wiener 
measure. Let B be the canonical process: B s (uj) — uo Sl s G [0, T], uj G fi. We denote by F = {F s , < s < T} 
the natural filtration generated by {B t }t>o and augmented by all P-null sets, i.e., 

F s = a{B r ,r < s} VA/>, sG[0,T], 

where Afp is the set of all P-null subsets and T is a fixed real time horizon. For any n > 1, \z\ denotes the 
Euclidean norm of z G M™. We introduce the following two spaces of processes which will be used later: 

S 2 (t 0} T; R n ) is the set of M"-valued F-adapted continuous process (ipt)t <t<T which also satisfies 
E[ sup \ipt\ 2 } < +oo; H 2 (t 0} T;R n ) is the set of M™-valued F-progressively measurable process (ipt)t <t<T 

t a <t<T 

which also satisfies || ip || 2 = E[f^ \i(jt\ 2 dt] < +oo; where to G [0,T]. 

First we introduce the setting for stochastic optimal control problems. We suppose that the control 
state space U is a compact metric space. U is the set of all U-valued F-progressively measurable processes. 
If u G U, we call u an admissible control. 

For a given admissible control u(-) G U, we regard t as the initial time and £ G L 2 (il, Tt, P; K") as 
the initial state. We consider the following fully coupled forward-backward stochastic control system 

' dX* a >t> u - b(s,Xt& u ,Yf& u , Z t /' u 7 u s )ds + a(s,X t /' u ,Y s t ^ u ,Z t /' u ,u s )dB s , 
dY^ u = -/( S ,X*^«,y s ^",Z*^«, Us )rfs + ^"rfS s , se[t,T], 

xp* = c, (2J) 

Y^ C;u = $(^ C;u ), 

where the deterministic mappings 

b : [0, T] x M™ x R x R d x U — > M™, a : [0, T] x R" x M x R d x U — > R" xd , 

/ : [0, T] x R™ x R x R d x U — > R, $ : R" — > R 

are continuous in (t,u) G [0, T] x U. In this paper we use the usual inner product and the Euclidean norm 
in R™,R m and R mxd , respectively. Given an m x n full-rank matrix G, we define: 

-G T f 

A= | u | . H/.A) = | G6 | (i,A), 




where G T is the transposed matrix of G. 
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We assume that 

(Bl) (i) A(t, A) is uniformly Lipschitz with respect to A, and for any A, A(-, A) G 
% 2 (0,T;R™ x R m x W nxd ); 
(ii) $(x) is uniformly Lipschitz with respect to x G R n , and for any x G K", $(x) G 
L 2 (0, J" T ,R m ). 
The following monotonicity conditions are also necessary: 

(B2) (i) (A(t,X)-A(t,X),X-X) <-^\Gx\ 2 -f3 2 (\G T y\ 2 + \G T z\ 2 ), 

(ii) ($(x) — 5>(af), G(x — x)) > /xi |Gx| 2 , x = x — x, y = y — y, z — z — z, 
where fli, f3%, \i\ are nonnegative constants with fix + j3 2 > 0, + (i\ > 0. Moreover, we have /3i > 0, /xi > 
(resp., /9a > 0), when m > n (reap., m < n). 

Remark 2.1. (B3)-(ii) ($(x) - $(x),G(x - x)> > 0. 

The coefficients satisfy the assumptions (Bl) and (B2), and also 
(B4) there exists a constant K > such that, for all £ G [0,T], u elf, Xi, x 2 G R™, yi, j/a G R, Zi, 2 2 G R d , 

|Z(£,xi,yi,zi,u) - l(t,x 2 ,y 2 ,z 2 ,u)\ < K(\xi - x 2 | + \yi - 2/2 1 + \zi - z 2 \), 

I = b,a,f, respectively, and |$(xi) — &{x 2 )\ < K\x\ — x 2 \. 

Remark 2.2. Under our assumptions, it is obvious that there exists a constant C > such that, 

\b(t,x,y,z,u)\ + \a(t,x,y,z,u)\ + \f(t,x,y,z,u)\ + |$(x)| < G(l + |x| + |y| + |z|), 

for all (t, x, y, z, u) G [0, T] x R" x R x R d x U. 

Hence, for any u(-) G U, from Lemma 2.4 in jjJJ, FBSDE ()2.1j) has a unique solution. 
From Proposition 6.1 in [8], there exists G G R + such that, for any t G [0, T], f, C G L 2 (0, JF t , P; R"), 
it(-) G W, we have, P-a.s.: 

(i) E[ sup [X s ^ u -X s *^ u [ 2 + sup [Y?.fc» -r s *^'-"| 2 + C\Z l s ^ u -Z^'^ds I Ji] < G|C-CT, 

t<s<T t<s<T 

(ii) £[ sup |X^-"| 2 + sup \Y^ U \ 2 + jf \Zl^ u \ 2 ds I Ft] < G(l + (CI 2 )- 

t<s<T t<s<T 

(2.2) 

Therefore, we get 

(i) |Y«>«|<G(l + |C|),P-a.s.; 

(ii) |y/^-y t *^ tl |<G|c-CI,P-a.s. 1 ' ' 

We now introduce the subspaces of admissible controls. An admissible control process u = (ur)re[t,s\ 
on [£, s] is an F-progressively measurable, [/-valued process. The set of all admissible controls on [£, s] is 
denote Ut, S) t < s <T. 

For a given process u(-) G Ut,T, we define the associated cost functional as follows: 

J(£, x; u) := Y l s < x > u | s=t , (£, x) G [0, T] x R™, (2.4) 

where the process Y l > x ' u is defined by FBSDE (|2.1I) . 

From Theorem 6.1 in g] we have, for any t G [0,T] and C G L 2 (fi, T tl P\ R"), 

J(£,C;n) = F t *' C; ™, P-a.s. (2.5) 

For £ = x G R™, we define the value function as 

W(t,x) := esssup ueWt J(t,x;u). (2.6) 
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Remark 2.3. From the assumptions (Bl) and (B2), the value function W(t,x) as the essential supremum 
over a family of ~F t -measurable random variables is well defined and it is a bounded J- \ -measurable random 
variable too. But it turns out to be even deterministic. 

In fact, inspired by the method in Buckdahn and Li [2], we can prove that W is deterministic. 

Proposition 2.1. Under the assumptions (Bl) and (B2), for any (t,x) G [0,T] x M™, W(t,x) is a deter- 
ministic function in the sense that W(t,x) — E[W(t, x)], P-a.s. 

The proof can be consulted in Li and Wei [8], Proposition 3.1. 

□ 

From ()2.3[) and ()2.6[) we get the following property of the value function W(t,x): 
Lemma 2.1. There exists a constant C > such that, for all < t < T, x, x' G R n , 

(i) \W(t,x)-W(t,x')\<C\x-x'\; 

(ii) \W(t,x)\<C(l + \x\). 1 ■'> 

Lemma 2.2. Under the assumptions (Bl) and (B2), the cost functional J(t,x;u), for any u £ Ut,T, om-d 
the value function W(t, x) are monotonic in the following sense: for each x, x G R n , t G [0,T], 

(i) (J(t, x; u) — J(t, x; u), G(x — x))>0, P-a.s.; 
(h) (W(t,x) - W(t,x), G(x-x)) > 0. 

For the proof the reader is referred to Lemma 3.3 in Li and Wei [8]. 

Remark 2.4. (1) From (B2)-(i) we see that if a doesn't depend on z, then @2 = 0. Furthermore, we assume 
that: 

(B5) the Lipschitz constant L a > of cr with respect to z is sufficiently small, i.e., there exists some 
L a > small enough such that, for all t G [0,T], u G U, x\, X2 G M™, y\, 1J2 G M, z\, z^, G M d , 
\o{t,x 1 ,y 1 ,zi,u) - a(t,x 2 ,y2,Z2,u)\ < K(\xi - x 2 \ + \yi — 2/2 1) + L a \z\ - z 2 \. 
(2) On the other hand, notice that when a doesn't depend on z it's obvious that (B5) always holds true. 

The notation of stochastic backward semigroup was first introduced by Peng [17] and was applied to 
prove the DPP for stochastic control problems. Now we discuss a generalized DPP for our stochastic optimal 
control problem (|2.ip . (|2.6p . For this we have to adopt Peng's notion of stochastic backward semigroup, and 
to define the family of (backward) semigroups associated with FBSDE (12. ip . 

For given initial data (t, x), a real < i5 < T — t, an admissible control process u(-) G Ut.t+s and a 
real-valued J-t+s ® £>(R n )-measurable random function ^ : il x 1" — > M, such that (B2)-(ii) holds, we put 

&Z£ s m+ ■ y " ,: "- s€[t,t+s\, 

where (Xl' x ' u , Y*' x ' u , Z* ,a:; ")t< s <t + a is the solution of the following FBSDE with time horizon t + 5: 

dX^ x ' u = b(s, X l s ^ u , Y^' u , Z l / X j u , u s )ds + a(s, X l s ^ u , Y*' x;u , Z\^ u , u s )dB s , 

jytw = -f( s ^xl^ u ,Yt< x > u ,Zl' x > u ,u s )ds + Zl> x < u dB s , se[t,t + 6], 

X\^ u = x, 

Y t %T - *(t + 6,X^n- 

Remark 2.5. (i) From Lemmas 2.4 and 2.5 in we know that, if ^ doesn't depend on x, FBSDE \2. 8\) 
has a unique solution {X l ^ u , Y t > x ' u , Z l ' x > w ). 

(ii) We also point out that if ^ is Lipschitz with respect to x, FBSDE \2. 8\) can be also solved under the 
assumptions (B4) and (B5) on the small interval [t,t + S], for any < S < So, where the small parameter 
Sq > is independent of (t, x) and the control u; see Proposition 6.4 in 



Since $ satisfies (B2)-(ii), the solution (x t ' x ' u ,Y t ' x ' u ,Z t ' x ' u ) of FBSDE §3} exists and we get 

yt.X'.UTx, / -\rt,X\UX\ ryt.X'.U \\rt.X;U-\ 



Moreover, we have 



J(t,x;u) = Yt' x ' u = G^mx^)} = G^\Yt£ u ] = G%$[J(t + ^X^u)}. (2.9) 

Theorem 2.1. Under the assumptions (Bl), (B2), (B4) and (B5), the value function W(t,x) satisfies the 
following DPP: there exists a sufficiently small Sq > 0, such that for any < i5 < <5o, t G [0,T — 6], x G R", 

W(t,x)= ess sup „ eWtt+3 [W(t + 5, Xffi)] • 
The proof refers to Theorem 3.1 in Li and Wei |S]. 

Notice that from the definition of our stochastic backward semigroup we know that 

G t ^ s [W(t + 5,Xt$ u )}=Y^, se[t,t + 6}, u(.) G U t , t+S , 

where (Xl' x;u , Y*' x ' u , Zl' x ' u )t< s <t+S is the solution of the following FBSDE with time horizon t + 6: 

= b(s, X*> x '> u , Y*' x ' u , Zl' x ' u , u s )ds + a(s, X^ u , Y*> x ' u , Z\^ u , u s )dB s , 
= -f(s,x)^,Y*^,Z*^,u.)ds + ^ u dB B , se[t,t + S], 

■ t+ s = W(t + 6,X%n- 

From Proposition 6.4 in [5] there exists a sufficiently small do > 0, such that for any < 5 < So, the above 
equation (|2 . 10[) has a unique solution {X l ' x ' ,w , Y t ' x > v , Z t,x ' u ) on the time interval [t, t + 6]. 

From Lemma 2.2, we get the Lipschitz property of the value function W(t,x) in x, uniformly in t. 
Now from Theorem 2.1 we can get the continuity property of W(t,x) in t and to conclude: 

Theorem 2.2. Under the assumptions (Bl), (B2), (B4) and (B5) W(t,x) is continuous in t. 

Its proof can be found in Li and Wei jSj, Theorem 3.2. 



dX\' x ' u 

rfyt,x;u 



H 

'rt,X\U 



3 Viscosity Solutions of HJB Equations 

In this section we show that the value function W(t,x) defined in (|2.6[) is a viscosity solution of the 
corresponding HJB equation. For this we use Peng's BSDE approach [17] developed for stochastic control 
problems of decoupled FBSDEs. 

Let us consider equation (12. 1[) : 



r dxi^ u 

dY t,x;u 
\rt,x;u 

A i 

•yt,x;u 
T 



b{s, Xl' x ' u , Y^ x ' u , Zl' x ' u , u s )ds + a{s, Xl' x > u , Y^' x ' u , Z\> x ' u , u s )dB s , 



= -f(s, Xl< x ' u , Y*< x > u , Z\^ u , u s )ds + Zl' x ' u dB s , 
= x, 



G [t,T] 



The related HJB equation is the following PDE combined with the algebraic equation: 



^-W(t, x) + H v (t, x, W(t, x)) = 0, 



dt 



V(t, x, u) = DW(t, x).a(t, x, W(t, x), V(t, x, u),u), {t, x) G [0, T)xR", u G U, 
W(T,x) = <f>(x), ier, 



where 



(3.1) 



(3.2) 



H v (t, x, W(t, x)) = sup{DW(t, x).b(t, x, W(t, x), V(t, x, u), u) + f(t, x, W(t, x), V(t, x, u),u) 
ueu 

+ ±tr(aa T (t,x,W(t,x),V(t,x,u),u)D 2 W(t,x))}, t £ [0,T], x G R". 

We give the definition of viscosity solution for this kind of PDE. For more details on viscosity solution 
refer to Crandall, Ishii, Lions [3J. 
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Definition 3.1. A real-valued continuous function W 6 C([0, T] x R") is called 

(i) a viscosity subsolution of equation \3.2^l if W(T, x) < <&(x), for all x G R", and if for all functions 
<p G Cf b ([0,T] X R n ) and for all (t,x) G [0, T) x R™ swc/i i/iai W ~ ip attains a local maximum at (t,x), 

^{t,x)+H i> {t,x,<p(t,x)) >0, 

where ip is the unique solution of the following algebraic equation: 
ip(t, x, u) = Dtp(t, x).a(t, x, ip(t, x),ip(t, x, it), u), u G U. 

(ii) a viscosity supersolution of equation A3.2\) if W(T,x) > $>{x),for all x G R", and if for all functions 
ip G Cf b ([0,T] X R") and for all (t,x) G [0,T) x R™ smc/i that W — if attains a local minimum at (t,x), 

( ^-(t,x)+H^(t,x,p(t,x)) <0, 

< where ip is the unique solution of the following algebraic equation: 
[ ip(t 7 x 7 u) — Dip(t 7 x).<7(t,x,ip(t 7 x 7 u),ip(t,x),u), U<EU. 

(iii) a viscosity solution of equation if it is both a viscosity sub- and supersolution of equation VS.ty) . 

Remark 3.1. When a depends on z we need the test function ip in Definition 3.1 satisfies the monotonicity 
condition (B3)-(ii) and also the following technical assumptions: 
(B6) fa > 0; 

(B7) Ga(s, x, y, z) is continuous in (s,u), uniformly with respect to (x,y,z) G R x R n x R d . 

Theorem 3.1. Under the assumptions (Bl), (B2), (B4), (B5), (B6) and (B7), the value function W is a 
viscosity solution of H3.S\) . 

We have the following important Representation Theorem for the solution of the algebraic equation. 

Proposition 3.1. For any s G [0, T], C G 2/ G R, x G R™, u G U, there exists a unique z such that 
z = £ + Dtp(s,x)o~(s,x,y + ip(s,x), z,u). That means, the solution z can be written as z — h{s,x,y,Q,u), 
where the function h is Lipschitz with respect to y, £, and \h(s,x,y,C,,u)\ < C(l + \x\ + \y\ + |£|)- The 
constant C is independent of s, x, y, £, u. And z — h{s,x,y,Q,u) is continuous with respect to (s,u). 

Proof. It's obvious that we only need to consider the case when a depends on z. Then we have (B6) and 
(B7) hold. Furthermore, following the proof of Proposition 4.1 in [8] we can prove it. □ 
Proof of Theorem 3.1. Following the proof of Theorem 4.2 in [8] we can prove it. □ 
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